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AliSl’RACT 


ll''frrencc  Is  a central  Issue  for  lanfluai’.c  comprehension  and  generation.  After 
rrvirwlng  existing  process  models  for  comprehending  and  generating  referring 
expressions,  we  present  a general  framework  for  context  and  reference  processing. 
Tile  context  for  reference  processing  is  represented  as  a "Public  Workspace." 
Ilefcrcnce  processes  access  Public  Workspace  and  modify  its  content,  which  is  the 
set  of  concepts  cm  rently  "on  the  table"  as  far  as  the  current  language  interaction  is 
concerned.  Information  from  many  different  sources  can  be  Integrated  in 
comprehending  or  generating  referring  expressions.  Within  this  general 
framework,  a new  system  for  selectively  generating  referring  phrases  is  developed. 
This  system  decides  how  much  to  express  about  a given  concept  in  a given  context. 
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INTRODUCTION 


WHlon  Times,  W.Va.s  "The  bad  news  didn't  surprise  Miss  Ankrom,  who  is 
rxpcctlnfi  a baby.  She  said  she  had  been  half  expecting  it." 


It  is  clear  that  context  plays  a critical  role  in  the  comprehension  and 
generation  of  languaj^c.  By  itself,  the  word  "it"  has  little  meaning— only  in  a 
particular  context  does  it  have  a referent.  Although  the  importance  of  context  is 
obvious,  models  of  language  processing  have  not  dealt  with  context  in  a 
satl.sfi'ctory  way. 

We  will  discu.ss  some  recent  process  models  of  the  comprehension  and  the 
generation  of  referring  phrases.  First,  we  will  look  at  the  various  representations 
of  context  that  have  been  proposed,  then  we  will  cover  the  various  ways  in  which 
context  has  been  used  in  the  processing  of  referring  phrases,  and  finally,  we  will 
present  a model  that  decides  how  much  to  include  in  the  referring  phrase  generated 
for  a given  concept. 
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TIIK  REPRESENTATION  OF  CONTFJCT 
FOR  REFERENCE  PROCESSING 


rhllosophcrs  of  lnnf,uaf,r,  linp.uists,  psychologists  studying  lan^^uafje,  and 
computer  scientists  building,  lanf;ua/;c  processinf,  systems  have  until  recently 
avoided  context  Issues.  One  rca.son  has  been  the  lack  of  fiood  repre.sentations  of 
context.  Wo  will  look  at  some  of  the  process  models  of  languafie  that  have  used 
context  to  deal  with  referring,  expressions  and  then  examine  some  of  the 
psycholo|*ical  evidence  of  the  structure  of  context  in  human  lanfiuai^e  processing. 
Using  some  of  the  notions  contained  in  recent  representational  formats  developed  In 
artificial  Intelligence,  we  will  describe  a new  proposal  for  representing  context  for 
reference  processing. 

We  are  presuming  a view  of  communication  In  which  sequences  of  words, 
tf (tiling  phraits,  are  a.ssoclalcd  in  the  nhiids  of  the  speaker  and  listener  with 
teftitnl  eoncrpis.  These  referent  concepts  are  often  also  a.ssociated  in  their 
mind.s  with  particular  objects  and  events  in  the  world  of  discourse,  called  the 
ttftirtU  objtcts  of  the  phrases,  rhra.ies  having  a common  referent  concept  are 
called  re  refettnltal. 

haiiguago  comprehension  requires  the  discovery  of  referent  concepts  for 
referring  phrases,  whereas  language  generation  requires  the  production  of 
referring  phra.ses  for  referent  concepts.  The  notion  of  reference  is  best  exemplified 
by  referring  phrase.s  which  are  syntactically  noun  phrases  or  pronouns,  and  by 
referent  concepts  which  represent  concrete  individual  objects  or  sets  of  such 
objects.  However,  referring  phrases  may  come  in  other  syntactic  forms,  and 
referent  concepts  may  repre.sent  events  and  abstract  concepts  as  well  as  concrete 
objects. 

lleference,  as  we  arc  viewing  it,  is  a procedural  notion  specific  to  Individual 
process  models;  one  cannot,  in  general,  speak  meaningfully  of  the  coirtc!  referent 
concept  of  a given  plira.se.  To  illustrate  this  consider:  "1  took  the  paint  off  the  shelf 
and  opened  it."  One  model  may  decide,  say  by  .syntactic  means,  that  the  referent  of 
"it"  is  the  r.rlnt  and  then,  considering  the  .semantics  of  "open,"  decide  that  what  wms 
oi'cned  was  not  the  paint  It.solf,  but  its  container.  A second  model  may  operate  by 
looking  through  its  context  for  a "genderle.ss  individual"  object  w'hich  can  he 
cipeued,  and  find  the  can  of  paint  as  the  referent  concept  of  "it."  The  two  models 
have  come  to  the  same  interpretation  of  the  sentence,  but  by  different  paths.  We 
are  distinguishing  the  question  of  "what  wms  opened,"  which  relates  only  to  the 
information  content  of  the  sentence,  from  the  question  of  "what  is  the  referent 
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concept  of  'it',"  which  m.iy  h.ivc  different  answers  for  different  processes 
Independent  of  how  they  answer  the  first  question. 

SVNTAX  DOMINATFD  CONTEXT  REPRESENTATIONS 

Thouf.h  it  remains  one  of  the  landmark  systems  for  processing;  natural 
lanp,ua/;c,  Blocksworld  (Wlnop.r.id,  ID/P.)  had  a relatively  simple  notion  of  context 
embedded  within  it.  For  comprehending,  and  f;eneratinp,  rcferrlni;  phrases, 
particularly  pronouns,  it  relied  heavily  on  searches  throuy.h  the  parse  trees  of  the 
pieviou.s  uttenmees.  Inforniatlon  al>out  the  syntactic  position  of  previously 
occurring,  referring,  phra.ses  stored  in  these  parse  trees  was  used  by  Blocksworld  to 
find  the  referent  of  a referring,  phrase.  This  context  was  dynamic,  in  the  sense  that 
only  the  few  mo.st  recent  parse  trees  were  searched. 

SFMANTIC-DOMINATED  CONTEXT  REPRESENTATIONS 

Another  approach  to  relerencc  proce.ssln/;  was  taken  by  the  MARGIE  system 
(Riei;er,  1D75).  lu  this  system,  the  com  prehension  of  a referring  phrase  was 
deferred  by  the  p.irsinf;  system,  and  was  attempted  instead  by  a later  Inferencitvg 
.system.  As  a conte.\t  within  which  to  pioce.ss  the  reference,  MARGIE  used  its 
memory  of  "conceptual  dependencies."  None  of  the  syntactic  information  about  the 
place  of  the  current  expression  in  the  input  utkorance  nor  syntactic  information 
about  previous  expressions  was  u.scd  by  the  inferencinp,  part  of  MARGIE.  Also,  its 
context  was  monotonlcally  ^rowlnf,,  modified  only  by  addition  of  conceptual 
dependency  structure  derived  from  the  comprehension  of  successive  input 
utteranco.s. 

Several  laitf.u.ip.e  comprehension  models  have  introduced  information 
.seheuias  which  amalgamate  knowledge  about  a central  concept  (Schank  8c  Abelson, 
in/Aj  Robrow  Winor.rad,  11)77;  Hayes,  1977).  Once  these  models  have 
determined  that  the  central  concept  has  been  referenced,  the  entire  schema  forms  a 
context  which  makes  related  concepts  available  for  instantiation.  One  purpose  of 
this  is  to  focus  the  search  for  referent  concepts  of  expressions  seen  later. 

PSVCIIOI  OCICAI.  EVIDENCE  AROUT  HUMAN  CONTEXT 

The  notion  of  context  that  we  have  been  exploring  corresponds  roughly  to 
the  p.<:ycholof,lcal  notion  of  "short-term  memory."  As  the  name  Implies,  humans 
have  a memory  for  events,  which  goes  away  after  a while.  Numerous  experiments 
have  investigated  the  nature  of  this  memory  u.sing  nonsense  syllables,  but  few 
have  u.sed  naturally  occurring  language,  largely  because  of  a lack  of  an  adequate 
repre.sentatlon  for  language  as  a stimulus  material.  However,  in  experiments  with 
non.scnsc  syllables,  short-term  memory  has  been  found  to  be  limited  (Miller,  1956) 
but  hierarchically  structured  (Mandicr,  19G8). 
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MUmi  FVH  RFI’RFSFNTATIONS 

Cousl.sUMit  with  thrso  lindlti/'.s  of  hicrjrcliical  structure  in  short-term 
iiuMUory,  several  rc'cent  represcntiitioiial  formats  have  incorporated  the  notion  of 
imiltilevel  descriptions  of  concepts.  The  MKItl.iN  sy.Ntem  (Moore  fic  Newell,  1973) 
and  later  the  KHL  sy.steiu  (Hohrow  ft  Winop.rad,  1977)  arc  representational  formats 
that  allow  concepts  to  be  represented  at  many  different  levels  of  specificity. 


77  Paso  Ttfifs,  Texas:  "The  niana^’er  of  a nudist  park  complains  that  a 
hole  wa.s  cut  in  the  wall  surrounding,  the  camp.  Police  are  looking*  into 
it." 


The  need  for  niulttple  levels  of  de.scriptlon  in  reference  processing;  is 
illustrated  by  this  quotation,  l.et  u.s  consider  the  issue  of  deterniinin/;  the  referent 
concept  for  "it."  The  humor  comes  from  the  incon^.rulty  of  the  interpretations  that: 
1 ) the  police  are  lookin/;  into  the  nudist  park,  or  ?.)  the  police  are  looking;  into  the 
hole,  or  3)  the  police  are  lookinr.  into  the  wall.  However,  note  that  even  when 
the.so  ineonp.ruous  interpretations  are  ruled  out.  there  remain  two  distinct 
Interpretations;  1)  the  police  arc  looking,  into  the  complaint,  or  3)  the  police  are 
looking,  into  the  cutting,  of  a hole.  We  are  not  Ininiedlately  aware  of  thl.s 
"ambl/;ulty"  bi'cause  it  doe.sii't  really  make  much  difference  for  the  overall 
interpretation  of  the  sentence.  At  some  level,  these  two  low-level  differences  in 
interpretation  are  covered  by  a common  hlf.her  level  de.scriptlon. 

I’ROI’FRTIFS  OF  CONTEXT  FOR  RFFFRFNCF  PROCESSING 

To  summari.’.e  the  di.scii.ssioii  so  far,  context  for  reference  processing;  requires 
the  following,  properties: 

1 ) it  is  derived  from  the  lanf.u.Tp.e  beliip.  processed, 

?.)  it  draws  upon  permanent  kiiowled^.c, 

3)  it  is  transient, 
d)  it  is  structured, 

6)  il  permits  multiple  levels  of  description, 
rum  !C  WORKSPACE 

Given  these  desiderata,  wo  have  developed  a rcpre.sentation  for  context  that  is 
derived  from  a previous  notion  of  ”Worksp.ice"  (Levin,  1976)  as  a kind  of 
structured  dynamic  short-term  memory  containing  "activations"  of  permanent 
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"concepts'*  and  relationships  between  them  in  lonp.-tcrm  memory.  Each  activation 
In  the  Workspace  has  a numerical  s.ilience  value;  when  the  salience  of  a given 
activation  falls  below  a global  threshold  level,  the  activation  disappears  from  the 
Workspace. 


"I’liMtc  Workspace"  Is  a subset  of  the  overall  Workspace,  containing  only 
those  activations  derived  from  the  processing  of  utterances  In  tlie  current 
Interaction.  In  a dl.ilogue,  for  example,  the  Public  Workspace  of  a participant 
contains  activations  of  those  concepts  and  relationships  "on  table"  as  far  as  the 
current  state  of  the  dialogue  }s  concerned.  Since  each  partlclpaiit  knows  that  the 
other  participant  Is  currently  aware  of  these  concepts,  each  participant  can  rely 
upon  this  shared  awareness  in  processing  referring  phrases,  Tlil.s  Public  Workspace 
representation  Is  similar  functionally  to  (Shale's  "foreground"  notion  (Chafe,  1974). 


6 


THE  USE  OF  CONTEXT  IN  REEERENCE  PROCESSING 


So  lar.  wr  hflvc  restricted  our  discussion  of  context  solely  to  the 
representation  Issues.  We  arc  also  Inlercsted  in  the  kinds  of  processes  that  use 
context  to  dejl  with  referring,  phrases.  In  this  section,  wo  will  re-examine  some 
natural  t nAua/tP  systems,  looKinp,  at  the  Kinds  of  processes  they  use,  and  we  will 
cla.ssify  these  proce.sses  into  two  major  cate/*ories,  which  we  will  relate  to  our 
ruhllc  Workspace  representation  of  context. 

The  alp.orithnis  used  for  reference  determination  are  examples  of  the 
p.eiierate-and-test  paradigm,  diverse  sorts  of  information  arc  p.enerally  available 
for  flndlnp.  a referent--syntactic  clues,  discourse  clues,  woru’  knowledy.e- -and  the 
alp.orithms  typlc.illy  use  certain  sorts  o!  information  to  gennar  possible  referent 
concepts  and  other  .sorts  of  information  as  filters  to  test  the  plausibility  of  the 
hypothesised  referent  concepts. 

KFKFRFNCF  PROC.FSSFS  IN  FXISTING  SVSTFMS 

Rrfrtrnrp  In  RIoeUwnrId 

VN'Inop.rad  (15)/?.)  Iniplemeuted  heuristics  for  reference  re.solution  which 
were  .il'ie  to  de.il  with  two  modes  of  reference  to  the  objects  in  his 
Hlocksworld;  reference  by  p.iriiai  dc.scnption  and  reference  by  pronoun.  For 
partial  dc.scrlptions.  a list  of  all  Idocks  that  had  tlie  properties  specified  in  the 
description  was  collected.  If  the  si?e  of  this  set  was  appropriate  for  the  quantifiers, 
qualifiers,  and  determiners  In  the  de.scnptum,  the  set  of  objects  found  was  u.sed  as 
the  referent.  If  too  many  were  found,  an  attempt  was  made  to  restrict  the  .set  to 
those  most  recently  referenced.  If  too  few  were  found,  the  description  wa,s 
reparsed  If  possible.  As  a last  resort,  the  system  asked  for  more  Information. 

The  process  f.enerated  pos.>iMe  referents  for  the  pronoun  "It"  not  by  uslny,  It.s 
.si’iiiantlc  model,  but  by  applying  rule  based  on  syntactic  criteria  for  finding; 
plauslbleco-referentlalexpre.sslon.s  .ui  tl.o  preceding,  dialo^.ue.  The  semantic  model 
of  the  lihKksworld  was  used  to  fiH'T  out  po.ssible  referents,  and  the  syntactlcallj' 
most  plausible  referent  concept  which  pa.ssed  the  semantic  filter  was  accepted. 
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Rrfrrriicf  in  MARCIK 

Some  systems  find  the  referent  concepts  for  expressions  solely  on  the  basis  of 
siMiiantic  features,  ip.norinf,  syntactic  features.  Quillian  (1969)  described  such  a 
system,  and  IUep,er  ( 1975)  used  a similar  approach  to  find  the  referent  concepts  for 
definite  noun  phra.ses  within  ‘.he  MAHGIE  System  (Schank,  Goldman,  Rieger,  St 
RIe.sbeck,  1973). 

In  Rieger's  approach,  a referring  expression  contributes  specifications  which 
the  referent  concept  must  satisfy.  A search  is  conducted  to  find  all  concepts  in 
ineniory  which  are  compatible  with  tlie  sj>ccification.s.  Recausc  the  comprehension 
.system  need  not  have  the  same  knowledge  about  the  discourse  objects  as  the 
speaker  has,  this  compatibility  test  involves  a partial  match  process,  which 
rniilributes  a .score  indicating  how  well  an  object  matches  specifications.  In 
addition,  Rieger  maintained  for  each  concept  two  attributes  which  aided  in 
refi'ience  re.solution.  RI.CKNCV,  which  liidic<ited  wlicn  the  concept  la.st  served  as 
the  referent  concept  of  some  lingul.stic  stn.r*urc,  and  TOUCH  El),  which  indicated 
woen  the  concept  was  last  accessed  in  rea.soniiig  processes.  The  reference  finder 
list'd  tl  .se  attrihute.s  to  define  an  'inplicil  notion  of  salience,  which  was  used  to 
help  resolve  references  which  the  partial  match  process  failed  to  resolve. 

In  addition,  Ric'ger's  model  was  able  to  use  its  inference  capabilities  to  find 
.specifications  not  derivable  from  the  referring  expression  itself.  There  were  two 
.sources  of  such  specifications; 

• Any  ar‘*c,Tnent  of  any  predicate  could  have  a "specifier  program"  associated 
with  it.  "his  was  a program  which  could  add  now  specifications  for  the 
argument  a.ssociatcd  with  it,  contingent  on  context  and  on  existing 
specifications  for  the  various  arguments  of  the  predicate. 

• The  general  Inference  mechanism  was  designed  so  that  It  could  work  with 
predications  mixing  particular  concepts  and  dc.scribcd  concepts  (sets  of 
specifications).  When  the  specifications  from  the  referring  expression, 
together  with  those  added  by  any  applicable  specifier  program,  were 
insufficient  to  uniquely  identify  a referent,  those  specifications,  plus  any 
other  features  common  to  all  remaining  candidate  referent  concepts, 
formed  a temporary  concept.  The  inference  rules  v/ere  then  permitted  to 
operate  as  though  this  concept  were  a valid  referent.  The  inference  rules 
might  then  add  further  specifications  to  this  concept  and  eventually 
narrow  the  set  of  candidate  referent  concepts.  If  Inference  processe.' 
halted  without  identifying  a definite  referent,  the  temporary  concept 
could  be  carried  along  until  additional  text  was  processed.  In  theory,  only 
the  need  to  produce  some  form  of  external  behavior  could  force  the  model 
to  decide  on  a referent. 
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Cl  ASSinCATION  OF  TIIF,  Rl  FERENCK  HEURISTICS 

l.ot  US  look  at  some  reference  heuristics,  including  those  we  have  discussed 
already,  plus  additional  ones  described  in  papers  by  WarnocK  (197R)  and 
llaraiiofsky  ( 1970). 

We  can  classify  them  Into  two  catey.ories:  those  dealing*  with  the  possible 
referent  concepts  and  those  deallnr*  . . ‘h  the  referring*  expression. 

The  heuristics  that  deal  with  a.spects  of  the  pos.slblo  referent  concepts  are  a 
divcr.se  lot,  covering’,  both  syntax  and  semantics.  One  thin;^  seems  to  be  a common 
fe.iturc--each  seems  to  reflect  contributions  to  the  salience  of  the  various  concepts 
in  awareness.  Some  of  the  heuristics  capture  what  Is  called  "focus";  others  reflect 
the  fact  that  concepts  In  awareness  are  temporary,  disappearing  If  they  aren't 
refreshed. 

1.  I'roximity!  The  closer  an  expression  Is  to  the  referring  expression,  the  more 
likely  it  is  to  be  co-referontial  with  the  referring  expression.  That  is,  the  more 
recently  a concept  has  been  reicrcnced.  the  more  likely  It  Is  that  the  concept  Is 
again  being  referenced. 

"1  can't  find  any  documentation  on  the  program.  1 have  a tape  here  at  Purdue 
and  1 can't  figure  out  what  format  i('s  in." 

The  "it"  refers  to  the  tape,  not  to  the  documentation  or  to  the  program. 

?..  Syntactic  role  in  the  sentence: 

P.a.  Siibject/Clb.iect/I'reposition  phrases:  The  referent  concept  of  the  syntactic 
subject  of  an  utterance  Is  more  likely  to  be  re-referenced  than  Is  the  referent 
concept  of  syntactic  object,  which  is  itself  more  likely  than  the  referent 
concept  of  a prepositional  phrase. 

"O:  The  output  comes  out  on  the  line  printer. 

L;  Throw /'f  away .. . " 

The  "it"  refers  to  the  output  (the  syntactic  subject  of  the  first  utterance)  rather 
than  to  the  line  printer  (the  syntactic  object). 

?.b.  Superordinate/subordinate:  Concepts  referenced  In  a superordinate  clause 
are  more  likely  to  bo  ro-reforencod  than  those  in  any  subordinate  clauses. 

"...  the  tape  that  file  is  archived  on  seems  to  be  a bad  tape.  We  can't  seem 
to  get  it  to  read  ..." 

The  "it"  refers  to  the  tape  rather  than  to  the  file  (in  the  subordinate  relative 
clau.se). 


?,c.  Topicalization;  Some  special  syntactic  constructions  (such  as  cleft 
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srntcnccs)  can  be  used  to  emphasize  one  element  of  an  utterance.  The  referent 
concepts  of  emphasized  expressions  are  more  likely  to  be  re-referenced  than 
those  of  unemphasized  ones, 

"...  there  is  a background  Job  running  here  that  checks  to  see  If  there  Is 
any  unsent  mall.  Oncei'r  finds  some,  if  tries  to  resend  It." 

The  two  "lt"s  arc  co-referential  with  the  topicalized  "background  Job"  rather 
than  with  the  "unsent  mall." 

3.  Centrality:  The  more  frequently  a concept  has  been  referenced  the  more  likely 
It  Is  to  be  referenced  again. 

"!•:  . . . Any  chance  1 can  recover  [file  name]  from  the  most  recent  system 
dump? 

O:  Probably,  let  me  look  for  it  and  get  back  to  you,  ok? 

1.;  Could  you  SNDMSG  to  me,  one  way  or  the  other?  1 won't  be  doing  anything 
about  it  tonight.  If  it  is  there,  1 will  be  forever  grateful  to  recover  it," 

The  "it"s  by  I.  all  refer  back  to  L's  file,  rather  than  to  the  system  dump  or  the  one 
way  or  the  other. 

d.  Current  topic:  An  expression  which  refers  to  a concept  In  the  current  topic  Is 
more  likely  to  be  co-referential.  Dcetsch  (lOVd)  observed  that  definite  references 
can  normally  be  made  only  to  concepts  that  are  part  of  a currently  "open  topic." 
Once  a topic  Is  "clo.sed,"  it  must  be  reopened  before  concepts  within  It  can  be 
referenced  again. 

"I,;  , . . Can  you  recover  those  files  for  mo..as  far  as  1 know  they  were  In  the 
directory  on  the  Itith. . . the  namesare  . . . 

[ Intervening  dialogue  ] 

O:  OK  I have  found  tAf  files  you  want.  1 will  retrieve  tAose  for  you  ..." 

In  the  second  utterance,  O initially  specified  the  files  in  some  detail,  but  once  the 
topic  was  re-established,  Ju.st  "those"  sufficed. 


I'herc  arc  al.so  heuristics  that  supply  information  about  what  the  referent  of 
the  referring  cxpre.s.slon  can  be.  These  heuristics  focus  on  the  information  content 
of  the  expression  and  its  immediate  context. 

1 . Specification  within  the  referring  expression: 

la.  The  pronoun  in  an  expression  often  specifics  that  the  referent  concept  be  a 
particular  number  and/or  gender. 

"How  do  / gpt  Hunoff  to  work?" 

The  pronoun  "1"  completely  jpccifies  the  referent  concept  to  be  the  speaker. 

lb.  The  head  noun  of  the  referring  expression  frequently  names  a class  of 
objects  or  events;  the  referent  concept  must  be  an  individual  within  that  class. 
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"I  h.ivc  found  tht  fiUi  ..." 

Ic,  Tho  other  modifiers  In  the  relerrliif,  expression,  such  as  adjectives,  relative 
clauses,  prepositional  phrases,  and  possesslvos,  often  further  specify  the 
ri’ferent. 

"I  liave  found  tht  files  sou  aie  ionitined  iiKuif  ..." 

Speelf  Ic.itlon  by  the  verb  le  the  .s.uiie  utler.uice:  The  v«'rb  havlnf.  the  referring, 
expression  as  an  avp.uinent  often  specifies  th.it  the  relerent  concept  h.ave  cert.iin 
piopertles. 

"flirow  it  aw.iv," 

Till'  verb  (.ind  verb  particle)  "throw  .iw.iy"  specifies  the  referent  to  be  something,  of 
llUle  v.ihie  to  the  speaker. 

a .Sp,  I II  icatinii  b\  the  whoU'  danse  containiny,  the  K'lerilny,  expression: 
.Soinel’.nies  the  paiiiml.ir  conibin.itiun  o(  a verb  and  its  noun  phrase  arp.uments  puts 
innsti  .lints  on  the  relerent 

' iHd  prod  111  e any  output  tih  V" 

Hie  coinblnatton  ol  the  veib  and  the  olMcct  limit  the  relerent  to  beliif^  a coinpnter 
prop.r.iin  ol  .some  kind. 


A Gl'.NIsHAL  FKAMKWOKK  FOR 
RF.FFHI;NCF  COMPREHENSION 


Thr  cln.'jsiliciilion  of  the  reference  heuristics  Into  these  two  categories 
Miggc'sts  a general  approach  toward  modeling  reference  abilities  In  process  models. 
Theie  should  be  two  parts  ol  such  a model:  the  proce.sses  that  contribute 
.spi’cll ications  to  the  referent  concept  oi  the  current  expre.ssion,  and  the  processes 
that  aifect  the  salience  ol  all  the  other  currently  active  concepts.  Given  a 
spK'ading  activation  processing  Iraiiicwork  (Collins  & hoflus,  1976;  Levin,  1976; 
Levin,  1978),  we  can  Integrate  all  of  the  inforiuntion  supplied  by  the  heuristics 
enneerning  aspects  of  po.ssible  n-fereiit  concepts  by  having  each  such  process 
modify  the  salience  levels  ol  the  activations  In  the  Public  Workspace.  Since  these 
ch.uiges  are  glob.il,  the  resultant  salience  values  are  thus  the  integrated  result  of  all 
the  currently  active  proces.ses.  These  salience  values  can  then  be  used  to  resolve 
ambiguous  cases,  In  a way  analogous  to  but  more  general  than  the  plausibility 
rating.s  used  by  previous  systems.  With  this  mechanism,  any  additional 
Iniormatlon  sources  can  be  added  easily  b.v  having  them  modify  the  salience  of 
atrtivations  in  the  Public  Workspace,  concurrently  with  the  proce.sses  already 
Implemented. 

Initially,  the  relerent  concept  of  an  e,\pre.ssion  is  completely  unspecified. 
L.uh  of  the  heuristics  conceriiiiig  the  referring  expre.ssion  can  be  seen  as 
CHiiitrlbuting  specif  Icatlons  to  this  umspecilied  concept.  Kach  heuristic  may  operate 
Imlependently  in  adding  Its  constraints  to  the  referent  of  the  current  expression. 
The  end  result  is  the  partially  specified  relerent. 


iM!r::iriiii[|[i.»u:;,itiiii[iiii[i[ij, giBjg!iii.3g:i-  ..,iig!iia;..ji!saM  a 
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An  isr.uc  cln.st'ly  rrlntcd  to  rofcn-ncr  dctcrminntion  is  that  of  ellipsis,  a surface 
form  which  omits  the  specif ication  of  certain  syntactic  or  semantic  roles.  Most 
models  of  lanp.uap.e  comprehension  attempt  to  "recover"  omitted  material  in  at  least 
.some  situations,  that  is,  to  find  the  referent  concept  of  an  "empty"  expre.ssinii. 

A llnAul.stic  ai'proach  to  this  prohlein  is  to  view  ellipsis  as  the  result  of 
di'letion  rules  in  a A'animar,  and  to  recover  the  omitted  concept  by  reconstructing 
the  parse  tree  from  which  the  delelion  was  made.  The  result  is  to  determine  that 
the  empty  expression  is  co-releumtlal  with  some  expre.ssion  appearing,  in  the 
surface  structure,  to  which  the  normal  rules  for  reference  determination  can 
apply. 

The  /’,r<iJhniar- based  approach  offers  no  solution  to  many  forms  of  ellipsis, 
however.  For  instance,  it  c.uuiot  determine  the  mi.ssinf,  ob.lect  of  "fired"  in 

Tlie  soldier  picked  up  his  rifle,  lie  fired  <empty>. 

A straip.litforvvard  approach  to  finduip.  the  referent  in  such  a ca.se  is  to  treat 
<empty>  as  a iiiimbei le.ss,  f^emlci  less  pronoun  and  apply  available  rules  for  finding 
the  referent  concepts  of  pronouns,  it  is  not  clear,  though,  when  grammatical  rules 
f(u  pronouns  can  be  extended  to  omitied  arp.uinents,  or  to  what  extent  the  ob.served 
pi  ohahilitie.s  for  locations  of  antecedents  extend  to  ca.ses  of  ellipsis. 

One  reason  that  process  models  attempt  to  find  or  create  referent  concepts  for 
omitted  material  is  that  these  referent  concepts  may  themselves  be  the  referent 
concepts  of  cxprc.ssions  appearing  later  in  a text: 

The  .soldier  took  careful  aim  and  fired. 

The  rifle's  recoil  knocked  him  down. 

Moth  the  proximity  and  syntactic  role  rules  arc  enhanced  by  the  recovery  of 
omitted  arguments. 

It  has  also  been  noted  that  once  a concept  has  been  referenced,  certain 
concepts  predictably  associated  with  that  concept  can  serve  as  referent  concepts  of 
later  expre.ssions  without  any  explicitly  mentioned  link  to  the  first  concept.  This 
form  of  reference  has  been  handled,  at  least  in  some  cases,  by  the  introduction  of 
knowledge  schemas,  as  prevlou.sly  described.  When  a text  presents  sufficient 
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evidence  to  "instantiate"  one  of  these  units,  various  concepts  within  the 
instantiation  become  available  as  potential  referent  concepts,  even  though  they 
have  never  been  explicitly  mentioned  in  the  text.  Using  a "script"  about 
restaurants,  SAM  (SchanK,  1975)  can  process: 

John  went  to  a restaurant.  The  hostess  seated  John. 

and  interpret  It  in  a way  that  connects  the  referent  concepts  of  "restaurant"  and 
"hostes.s"  in  a rational  manner. 
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HEKERENCE  AND  GENERATION 


In  comparison  with  the  lar/*,c  Investment  in  process  models  of  language 
compiohonslon,  little  study  has  been  devoted  to  the  process  of  language  generation. 
This  is  particularly  true  with  respect  to  reference.  The  reference  problem  for  a 
generator  can  be  stated  as 

Given  the  goal  of  expressing  something  about  a concept  C in  a particular 
context,  how  should  C be  referred  to?  That  is,  what  subset  of  the 
Information  Xnown  about  C should  be  expressed  and  flow  should  it  bo 
expre.ssed? 

It  Is  possible  to  generate  phra.ses  to  refer  to  concepts  of  all  sorts:  specific 
physical  objects,  events,  abstract  qualities,  generic  objects,  sets,  predicates,  etc. 
The  natural  language  generator  has  two  orthogonal  concerns.  It  must  determine 
what  information  to  convey  about  the  concept,  and  it  must  select  the  words  and 
syntactic  forms  appropriate  to  carry  out  the  reference.  The  choice  of  information 
does  not  neces.sarlly  dictate  the  words  and  syntax: 

1 saw  f-'oid  aiui  Cfievy  collide. 

1 saw  f Ar  collision  beiween  the  Foul  (J«(f  the  Chev'^, 

Nor  does  a choice  of  syntax  nece.s.sarily  dictate  words  or  information  content. 

1 .'i.iw  the  rOfVuron  between  the  Ford  and  the  Chevy. 

1 saw  Xhr  avtomobilf  Clash. 

Till'  gnnerator  not  only  imist  produce  grammatical  referring  phrases  for 
Individual  concepts,  but  must  maintain  grammaticality  when  those  phia.ses  are 
roiuhined.  Certain  choices  may  constrain  others.  For  example,  "I  assume  the  Ford 
and  Chevy  collided"  Is  grammatical,  while  "I  assume  the  collision  between  the  Ford 
and  Chevy"  Is  not. 

In  generating  a ri’ferring  expre.ssion,  a generator  may  make  use  of  the 
previously  described  aspects  of  such  expressions:  explicit  specifications  within  the 
phrase,  specif ications  Implicit  in  the  .syntactic  role  played  by  the  phra.se  with 
respect  to  a verb,  and  specifications  implicit  in  a whole  clause.  This  means  that  the 
generator  may  be  able  to  make  traded  fs  within  a sentence;  the  selection  of  a 
particular  verb  or  the  Inclusion  of  certain  specifications  In  one  referring 
expre.ssion  may  affect  the  information  content  needs  of  another  reference. 


I 


k 


rnocKss  modei^ 


I.') 


It  1.";  po^.slblo  for  a verb  or  phr.rsp  to  contribute  implicit  specifications  to  a 
preceilinp,  phrase,  so  a strictly  le(t-to-riy.lit  y.eiieration  strategy  cannot  account  for 
some  reasonable  references.  Consider,  for  example,  the  common  form  of  solitaire  in 
which  a lep.al  niove  consists  of  placing,  a card  onto  another  card  of  opposite  color.  In 
a .situation  with  one  kinp,  and  one  queen  of  each  color  available,  the  statement 
''I'lare  the  queen  on  the  Kin/;"  is  ambiy.uous,  whereas  both  "Place  the  red  queen  on 
the  klnp,"  and  "Place  the  queen  on  the  black  kinp."  unaniblp.uously  specify  the  same 
move. 

I XISTING  RKFFRI  NCFGFNI  RATION  MODI  I S 

Two  y.pneration  models  (Simmons  Slocinn,  inVP<:  (loldman,  inVh)  addrt’s.sed 
only  the  Issue  of  how  to  evj'res.s  as  an  laiy.li.sh  strinp.  the  information  that  they 
were  f.lven.  Doth  pve.suined  that  no  information  selection  was  required. 
Hlocksworld  (Winnp.rad,  ID/P.)  was  de.si^'.ned  to  carry  on  a dialop.ue  and  thus  had  to 
deal  with  the  selection  aspect  of  reference  y.eneration.  The  ta.sk  domain  allowed 
three  simplifyln/’,  assumptions: 

• All  information  known  to  the  /generator  is  al.so  known  to  the  listener. 
Thus,  any  oh.tect  which  need.s  to  be  referenced  can  be  nderenced  by 
selecliny,  sufficient  information  titsdu^mshin^  it  from  other  objects. 
Fiirtherinori?,  any  piece  of  inloiniation  which  can  discriminate  between 
objects  In  the  robot's  model  will  enable  the  listener  to  perforin  the  same 
disci  inilnatioii. 

• The  number  of  ob.iects  in  the  world  model  at  any  time  is  sufficiently  small 
th.it  it  Is  conii'Utatioiially  feasible  to  determine  whether  a description 
nnhjuely  describes  an  ob.iect.  Tims,  it  is  not  neci'.ssar)'  to  maintain  a 
contr'xt,  or  sul'uiodel,  within  which  to  make  the  disci iniliiation. 

• Tlie  information  about  each  object  is  sufficiently  small,  and  sufficiently 
rey.ular,  that  the  selection  pioce.sscaii  avoid  search.  In  Itlocksworld,  this 
aiiioniited  to  selectinp,  properties  in  a fixed  order  until  the  accnmnlated 
properties  uniquely  described  the  object. 

Winoy.rad's  proy.rani  produces  both  definite  and  indefinite  noiin  phrases  to 
refer  to  toy  blocks.  The  choice  between  definite  and  indefinite  is  made  on  the  ha.sr>s 
of  syntactic  features  of  the  question  heiiiy,  answered.  The  noun  phrase  is 
constructed  by  concatenating,  adjectives  iiaiiiinp.  the  block's  sii';e  and  color  with  a 
noun  iiauiinp,  its  shape.  In  cases  where  a definite  reference  is  required  and  sise, 
.shape,  and  color  are  insufficient  to  uniquely  identify  the  block  being  referenced, 
relative  clau.ses  can  be  added  which  give  the  block's  proper  name,  if  it  has  one,  and 
Its  spatial  relationship  toother  blocks. 


I'liOCKSS  MODF.I^ 


in 


A fi'W  discourse  heuristics  in  lUocksworld  affected  the  generation  of 
referring  expressions,  Sets  of  individual  blocks  were  normally  referred  to  by 
conJt>inlng  the  descriptions  of  the  individuals  in  the  set.  However,  If  the 
descriptions  were  identical,  which  could  happen  when  Indefinite  descriptions 
Were  used,  the  entire  set  was  de.scribcd  by  prefixing  its  cardinality  to  a description 
of  one  of  the  elements  and  making  the  noiin  plural — "ihrec  small  cubes."  Pronouns 
would  be  Introdticcd  If  the  concept  being  referenced  had  served  as  the  referent  of  a 
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A SYSTEM  FOR  THE  SELECTIVE  GENERATION 
OF  REFERRING  PHRASES 


In  this  section,  we  will  describe  a system  we  have  developed  that  uses  the 
notions  described  in  the  previous  sections  to  tackle  some  of  the  problems  of 
deciding,  how  much  to  say  about  a given  concept.  In  a typical  generation  task,  we 
want  to  express  a given  concept  in  service  of  some  particular  generation  goal.  But 
there  Is  then  an  issue  of  how  much  needs  to  be  said  about  the  concept.  It  is  almost 
never  appropriate  to  express  everything  known  about  the  concept.  Obviously,  the 
choice  of  what  to  say  will  depend  upon  the  context  within  which  the  referring 
phrase  will  exist. 

Grice  (1975)  has  specified  at  a very  general  level  some  of  the  rules  for 
deciding  what  to  say,  in  a form  called  "conversational  postulates."  Of  particular 
concern  for  us  is  his  "I’ostulate  of  Quantity": 

1.  Make  your  contribution  as  informative  as  is  required  (for  the  current 
purposes  of  the  exchange). 

?..  Do  not  make  your  contribution  more  informative  than  is  required.  (Grice, 
1P75,  p.  d5) 

Wo  have  implemented  a system  that,  given  a particular  concept  to  express  and 
a particular  context,  creates  recommendations  of  what  aspects  of  the  concept  to 
express  and  what  aspects  not  to  express.  This  system,  called  CES,  represents 
context  as  a Public  Workspace,  as  already  described,  and  contains  processes  for 
using  the  salience  of  the  activations  in  Public  Workspace  and  the  specifications  of 
the  current  concept. 

WHAT  TO  SAV 

Consider  a particular  concept  to  express.  In  the  movie  The  Tall  Blond  Man  with 
One  Black  Shoe  there  is  a character  who  is  a tall  blond  man  with  one  black  shoe  (and 
one  brown  shoe).  Each  time  that  we  talk  about  this  man,  we  don't  want  to  have  to 
mention  all  of  the.se  specifications;  yet  we  usually  have  to  mention  at  least  some  of 
them.  This  is  about  as  far  as  the  Postulate  of  Quantity  take  us,  which  isn't  far 
enough  to  implement  a process  model  of  deciding  what  to  say.  We  have  to  know  in 
a bit  more  detail  how  to  determine  what  is  just  enough  to  say,  not  too  little  or  too 
much. 
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Suppose  the  context  is  set  by  the  comprehension  of  the  sentences  "The  tall 
blond  man  with  one  black  shoe  was  unarmed,  lie  was  chased  by  the  handsome  dark 
haired  spy  with  a p.olden  /*,un."  Within  the  framework  for  reference  presented 
here,  this  means  that  the  rubllc  Workspace  of  the  Generator  (and,  presumably,  also 
the  Hearer)  contains  an  activation  of  the  blond,  with  specifications  of  tall,  blond, 
man,  havinp,  one  black  shoe,  and  beinp,  unarmed,  and  also  an  activation  of  the  spy, 
with  specifications  of  hand.soiiie,  dark  haired,  spy.  and  having  a golden  gun. 
Suppose  that  we  now  want  to  generate  a .senience  saying  that  the  dark  haired  spy 
slipped.  We  can't  Just  say  "lie  slipped"  because  "he"  doesn't  distinguish  between 
the  tal'  blond  and  the  spy.  It  would  be  overkill  to  sa.v  "The  handsome  dark  haired 
spy  with  the  golden  gun  slipped."  A marginal  c.vpre.ssion  in  this  contex't  is  "The 
handsome  man  slipped,"  since  we  don't  know  whether  the  tall  blond  is  handsome  or 
not.  A better  expression  is  "The  dark  haired  man  slipped."  Here  we  start  to  see  some 
of  the  factors  delernilnlng  the  .selection  prore.ss--we  want  to  say  at  least  enough  to 
distinguish  the  concept  we  want  to  convey  from  the  others  in  Public  Workspace. 
This  is  natural  if  we  consider  the  piupo.so  ol  generating  the  expression:  to 
"stimulate"  an  activation  of  this  concept  in  the  workspace  of  the  Hearer.  This  goal 
can  be  achieved  only  if  we  supply  enough  information  to  rule  out  the  other 
concept.s  currently  "on  the  table"  in  the  conversation. 

Consider  another  sentence  we  might  want  to  generate  w’ithin  this  same 
roiitext.  We  want  to  say  that  the  spy  threatened  to  shoot  tl'.c  blond.  In  this  case, 
the  comprehender  ol  the  verb  "shoot"  in  this  sentence  can  generate  an  expectation 
that  the  person  shooting  has  a gnn.  So  it  is  not  necessary  to  express  the 
specifications  that  the  spy  had  a gnn.  Moreover,  wo  can  u.se  this  expectation  (and 
any  others  wo  have  for  the  po.sition  in  the  surface  senience  where  the  phrase  will 
appear)  to  rule  out  pnsslMe  ambiguities,  even  hefpte  anything  Is  expressed.  Thus,  in 
this  case,  we  only  need  to  rule  out  the  Inaiiimate  ob.iecls  to  convey  the  spy--saying 
"lu’"  is  sufficient. 

In  addition,  tlie  first  part  of  the  sentence  "He  threatened  to  shoot..." 
generates  expectations  for  the  other  parameter  of  "shoot"  that  uniquely  specify 
that  the  tall  blond  is  Intended,  evt'ii  before  anylhinr,  is  said.  So  we  need  not  say 
anything:  "He  threatened  to  shoot."  in  this  way,  wo  can  generate  ellipsis  In  cases 
allowed  by  syntactic  considerations. 

Tlie  CKS  Sy.stein  recommends  expre.ssing  the  specifications  of  the  current 
e.vpre.sslon  that  rule  out  any  "confusion  candidates"  in  the  rublic  Workspace.  A 
confusion  candidate  is  an  aciiv.itioii  ni  the  I’liblic  Workspace  that  is  con.sistent 
with  everything  known  so  far  about  the  current  expre.ss'on.  This  knowledge 
includes  the  c.xpectations  about  the  current  position  that  the  hearer  could  bo 
reasonabl,v  expected  to  share,  and  the  specifications  so  far  recommended  for 
expression. 
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Th<’  r\ j’ccf.uinns  ,it>out  tfic  current  position  nmy  be  y.ciierdtcd  by  the  other 
l.iiif,u,i/'.e  f,enor.ition  processes  runntnp.  concurrently  with  the  CKS  component.  For 
ev.niiple,  expert.Uions  c.in  Ix'  f.eiier.ited  about  the  current  position  from 
miiltisententlnl  structure  proce.s.se.s,  like  Scripts  (Sch.uik  & Abelson,  197vb)  or  the 
Ui.iloy.iie  (iaiiic  I’riHres.sor  (levin  Moore,  111//}.  Ctiireiitly,  however,  CES  collects 
i' V perfntlon.s  Irom  only  two  .sources,  lii.si,  it  looks  at  the  predicate  of  which  the 
current  concept  is  a parameter.  In  l.oiif.^Tei ni  Memory,  there  may  be  general 
iiiform.ition  Known  about  this  p.iraiiieter  position  which  can  then  be  expected  to 
hold  about  the  current  concept.  The.se  expectations,  plus  those  directly  supplied  by 
the  calling  proce.ss,  form  the  expectations  about  the  current  position. 

Older  of  Srirclloii 

In  the  proci'ss  of  con.sidcring  which  spociflcattons  to  express,  we  clearly  want 
to  .select  tho.se  that  rule  out  at  least  one  of  the  confusion  candidates.  However, 
thf-re  are  many  dilti  rent  po.ssible  sets  of  specifications,  each  of  which  would  rule 
out  all  the  confusion  candidate.s.  Fiiiiling  the  "optimal"  (.smallest)  set  would  be 
diKlrult.  Introducing  a combinatoric  explosion  problem.  For  the  CF,.S  system,  we 
h.ive  selected  an  approximation  to  an  optimal  choice.  At  each  point  In  the  process, 
the  CF-.S  system  si’lects  the  specification  to  expr.''.ss  that  rules  out  the  largest  number 
of  confusion  candidates.  This  proce.ss  continues  until  all  confu.slon  candidates  are 
eliniliiated. 

What  NOT  to  S.1Y 

The  CF.S  s.\  .st^'ni  decides  that  a specification  need  not  be  expre.s.sed  for  one  of 
two  different  re.i  ons.  First  of  all.  we  have  the  expectations  about  the  current 
position,  de.scribed  earlier.  Since  the.se  are  generated  from  common  know'ledge,  the 
hearer  probably  shares  the.se  expectations.  So,  any  specification  of  the  current 
roiicc’pt  that  Is  in  this  expectation  set  is  marked  by  CES  as  not  to  be  expressed,  since 
it  would  be  redundant  to  express  it. 

The  other  way  that  CES  decides  what  not  to  say  occurs  later  In  the  processing 
of  a given  concept.  CES  recommend.s  expressing  specifications  until  all  of  the 
confusion  candidates  arc  ruled  out.  Tho.se  remaining  need  not  be  c.xpre.s.sed. 

Fx.iiiiplr 

Eet  us  run  through  a quick,  example  of  how  the  CF,S  system  operates.  Given 
the  context  de.scribed  previously  for  the  tall  blond  man  and  the  spy,  how  does  CES 
operate  to  decide  how  to  express  the  handsome  spy  slipping?  First,  expectations  are 
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ccillrcUfi  from  Ihr  r.rncr.i!  concept  for  "sHp."  If  the  ap.cnt  were  constrained  to  bo 
animate,  CES  will  ns--  this  constraint  as  an  expectation  on  this  ap.cnt,  CES  now  will 
rule  out  all  Inanimate  concepts,  in  tins  case,  the  p,olden  p,dn  and  the  black  shoe.  To 
handle  the  case  of  expectation  violation,  as  in  metaphorical  uses  of  langua/;e,  if  an 
expectation  rules  out  the  concept  to  be  conveyed,  CK.S  will  Ip.norc  the  expectation. 
Itnt  CE.S  still  has  to  eliminate  the  reniaininp,  confu.sion  candidate,  the  tall  blond 
man.  The  specll Icatlon  of  dark-haired  is  selected,  because  It  explicitly  rules  out 
the  blond  man. 

In  a more  complex  case,  CES  will  try  each  specification  to  see  how  many  of 
the  confusion  set  are  ruled  out,  and  recommend  expressing,  the  one  that  rules  out 
the  most.  This  cycle  continues  until  no  more  confusion  candidates  ciflst,  or  no 
more  specifications  exist. 

In  .some  c.isi's.  CK«S  will  be  unable  to  rule  out  one  or  more  confusion 
ramlldates,  even  after  consldcrlnf,  all  known  specification  of  the  current  concept. 
Currently,  CES  pjves  up  and  admits  its  inability  to  uniquely  specify  the  concept, 
Entnrc  directions  for  rc.search  may  deal  with  these  problem  cases,  perhaps  by  use  of 
ti'xt  reference  ("the  second  spy  I mentioned"),  or  the  introduction  of  nonrestrictive 
specifications,  l.c.,  those  in  the  generator’s  private  Workspace  but  not  In  the  Public 
Vv'ork.space. 
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SUMMARY 


Wc  have  presented  a framework  for  reference  pro'^essing,  derived  from  a 
taxonomy  of  the  various  heuristics  currently  used  to  comprehend  references. 
Within  this  general  framework,  wc  described  a model  for  selecting  what  to  say 
(and  what  not  to  say)  when  generating  a referring  phrase  for  a concept.  The 
framework  proposes  that  the  various  sources  of  Information  bearing  on  reference 
processing  are  integrated  by  having  the  separate  processes  all  producing  global 
effects  on  one  common  Public  Workspace.  In  this  way,  many  different  processes 
can  be  defined  that  operate  autonomously  to  contribute  information  of  differing 
degrees  of  certainty,  which  arc  integrated  by  the  general  framework  presented 
here. 


Our  model  of  referring  phrase  generation  has  been  partially  Implemented  and 
tested  on  several  examples  for  a simple  data  ba.se.  We  believe  that  the  model  cannot 
be  as.scsscd  adequately  until  an  implementation  has  been  integrated  into  a text 
generation  process  capable  of  maintaining  the  Public  Workspace  data  base  across 
rhultlplc  utterances. 
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